Evaluating the Feasibility of RAG Models for Regulatory Compliance
Bridging AI and Economics
Jesús Martínez del Rincón
Abhishek Pramanick
Barry Quinn
Introduction
Collaborative Project Overview
- Led by Queen’s University Belfast researchers in the Finance and AI Research Lab - Industry Partner Funds Axis Ltd, a leading RegTech firm in Northern Ireland - Supported by UKRI through the UKFin+ programme.
- Aim: Evaluate the feasibility of Retrieval-Augmented Generation (RAG) models for enhancing regulatory compliance within FinTech.
Quantify economic efficiency through cost-benefit analysis and process mapping.
Address challenges in adaptability, interpretability, and usability.
Provide actionable insights for AI-driven compliance systems.
Towards a Unified Definition of AI
AI as Adaptive Capital
Artificial Intelligence (AI):
>A novel form of adaptive capital, capable of dynamic learning, autonomous decision-making, and task flexibility, designed to achieve specific objectives efficiently.
Key properties:
- Dynamic Efficiency: Self-improving through machine learning.
- Repurposability: Adaptable to diverse tasks.
- Scalability: Low marginal costs, enabling rapid expansion.
- Labour Dynamics: Both complement and substitute for human skills.
- Value Alignment: Ensures AI objectives align with societal and economic goals.
Repurposability: Application to diverse regulatory compliance tasks.
Scalability: Processes scale at low additional costs.
Labour Impact: Augments high-skill tasks; automates repetitive processes.
Value Alignment: Critical for sustainable economic growth.
graph TD
A[AI as Adaptive Capital]
A --> B[Dynamic Efficiency]
A --> C[Repurposability]
A --> D[Scalability]
A --> E[Labour Market Effects]
B --> F[Productivity Growth]
C --> F
D --> G[Reduced Costs]
E --> H[Redistribution of Skills Demand]
Implications for Policy and Economics
Aligning AI with Economic Goals
Value Alignment: Align AI’s objectives with human welfare.
Policy Challenges: Address labour displacement through upskilling and incentives.
Scalability & Risk: Leverage AI’s scalability while mitigating potential inequities.
Structural Changes: Long-term shifts in growth models and comparative advantages.
Methodology Overview
Hybrid Economic Analysis Framework
Process Mapping: Establish baseline workflows and resource allocation.
Cost-Benefit Analysis: Quantify direct and indirect cost savings.
Simulation Modelling: Test adaptability of RAG models to evolving regulations.
Stakeholder Feedback: Gather practical insights on usability and challenges.
Economic Methodology Overview
Hybrid Economic Analysis Framework
Preliminary Insights
RAG models are projected to reduce task time by 40%, saving ~20 staff hours weekly.
Early tests highlight improved interpretability for ESG reporting and risk assessments.
Stakeholders report an 8/10 usability score for compliance-related tasks.
• Focus: Fine-tune a regulatory-specific Q/A model. • Deliverable: First prototype tested for ESG compliance tasks.
WP3: Rule Extraction & Knowledge Base
• Focus: Automate rule extraction using OWL ontology. • Deliverable: Dynamic knowledge base with 500+ structured rules.
Challenges and Limitations
Quantified Challenges:
• Data Quality: Parsing accuracy starts at 70%, requiring preprocessing. • Adaptability: Simulations show initial 3-week lag for updates; goal is 5 days.
Addressing Concerns:
• Use explainable AI (XAI) for output transparency. • Focus on modular integration with existing systems.
Ethical Considerations
• Fairness: Minimise bias via curated datasets. • Accountability: Establish clear documentation for decision trails. • Human Oversight: Support compliance professionals without replacing roles.
Future Work
Train domain-specific language models for finance.
Enhance retrieval accuracy using semantic search.
Explore long-term cost and efficiency gains via real-world trials.
Thank You!
Contact Information Dr Barry Quinn Queen’s University Belfast 📧 barry.quinn@qub.ac.uk